Papers with text-to-image diffusion models
NeuroPrompts: An Adaptive Framework to Optimize Prompts for Text-to-Image Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | Recent advances in text-to-image diffusion models have made it difficult to obtain high-quality images. |
| Approach: | They propose an adaptive framework that automatically enhances a user's prompt to improve the quality of generation models. |
| Outcome: | The proposed framework generates prompts similar to those produced by human prompt engineers and provides user control over stylistic features via constraint set specification. |
UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion (2024.acl-long)
Copied to clipboard
| Challenge: | Existing text-to-image diffusion models generate images from text prompts due to inherent brevity of textual descriptions . however, the ability to accurately synthesize images with intricate details, such as specific entities or scenes, is limited due to the inherent bribery of text descriptions. |
| Approach: | They propose a multimodal conditional diffusion framework that operates on multimodal prompts with interleaved textual and visual inputs. |
| Outcome: | The proposed framework excels in both text-to-image generation and zero-shot subject-driven synthesis. |
Learning to Follow Object-Centric Image Editing Instructions Faithfully (2023.findings-emnlp)
Copied to clipboard
| Challenge: | avrahami et al., 2022b,a): natural language instructions are often underspecified, requiring models to uncover their implicit meaning. |
| Approach: | They propose to use paired data to model the implicit meaning of instructions . they also propose to ground the model to localize where the edit has to be performed . |
| Outcome: | The proposed model performs better than state-of-the-art baselines on paired data, showing improvements in quality and faithfulness. |